[WIP]add the confidence interval computation - #387
Open
YazhiGao wants to merge 1 commit into
Open
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
this is currently the implementation of ratio modeling for feature selection of random effect in photon.
I follow the algorithm described in the original publication but some twists are made according to discussion with yiming and alex.
Unit tests all pass.
Integration tests all pass.
The algorithm in reality(highly related with codebase instead of only mathematical expression) is as follows:
1.pass in the featureStatisticSummary
2.identify the binomial columns
3.compute the lowerbound for binomial columns based on the t value
4.select the feature based on only the following lowerbound criterion(non-binomial and intercept columns are kept automatically)
As a WIP commit, there are things to polish in near future since we currently focus on the feasibility of this experimental method and try to minimize user-side changes :
unit tests not fully covering all scenarios of feature selection. Currently the binomial cases are not selected, we need to craft some data that covering all cases.
binomial feature column identification predicate needs to be stronger. Current solution is inherently flawed, we need more computation at feature summary stage to ensure this one.
hyperparameter interface design. for convenience purposes, the user side interface for pass in normal distribution quartile and lowerbound threshold hyperparameter redesign.
the relationship with pearson correlation feature selection. We need another parameter to decide on the algorithm of feature selection or mix them in later stage.
crafted test data need some change, currently some unneeded feature summary entries are not carefully addressed.
further experiment report and benchmark report after regression tests
the way we currently keep non-binary and intercept columns is not good for further feature ranking report planned. need redesign
@joshvfleming @ashelkovnykov